Pictonaut: movie cartoonization using 3D human pose estimation and GANs

This article describes Pictonaut, a novel method to automatically synthetise animated shots from motion picture footage. Its results are editable (backgrounds, characters, lighting, etc.) with conventional 3D software, and they have the finish of professional 2D animation. Rather than addressing the challenge solely as an image translation problem, a hybrid approach combining multi-person 3D human pose estimation and GANs is taken. Sub-sampled video frames are processed with OpenPose and SMPLify-X to obtain the 3D parameters of the pose (body, hands and face expression) of all depicted characters. The captured parameters are retargeted into manually selected 3D models, cel shaded to mimic the style of a 2D cartoon. The results of sub-sampled frames are interpolated to generate a complete and smooth motion for all the characters. The background is cartoonized with a GAN. Qualitative evaluation shows that the approach is feasible, and a small dataset of synthetised shots obtained from real movie scenes is provided.This work is partially supported by the Spanish Ministry of Science and Innovation under contract PID2019-107255GB, and by the SGR programme 2017-SGR-1414 of the Catalan Government.Peer ReviewedPostprint (published version

Assessing the participatory design of a project-based course on computer network applications

New teaching methodologies which foster student involvement, such as project-based learning, are nowadays part of the study curriculum of many engineering schools. Project-based learning courses, however, often build upon other previously taught technical courses, where the technical content for the project to be developed is studied. That type of course design focuses on building the transversal capabilities of students, and the technical challenges of the project are the mean to acquire these non-technical skills. In this paper, we present and assess a project-based course on computer network applications of a computer science school, which has been designed to improve within the same course both the transversal and technical skills of the students. The proposition of interest is that the course not only aims to train the students’ transversal skills by a group work project, but also to practise new technical topics and technologies. We argue that the key element of the proposed course design is that each student project group defines with the instructor the project they would like to develop in the course. We present first the design of the course and then an assessment with questionnaires, which were conducted over two semesters with the students enrolled in the course. The obtained results indicate that the students achieved both technical and transversal skills, while the instructors need to be flexible to adapt to diverse technical topics of the proposed projects.Peer ReviewedPostprint (published version

Contorsion: A Semantic XPath Processor

AbstractThis work describes the architecture of Contorsion, a semantic XPath processor that acts over an RDF mapping of XML. It contributes to a recent research trend that defines an XML-to-RDF mapping allowing XML documents interoperate at the semantic level. We use a model-mapping approach to represent instances of XML and XML Schema in RDF. This representation retains the node order, in contrast with the usual structure-mapping approach. The processor can be fed with an unlimited set of XML schemas and/or RDFS/OWL ontologies. The queries are resolved taking in consideration the structural and semantic connections descrived in the schemas and ontologies. Such behaviour, schema-awareness and semantic integration, can be useful for exploiting schema and ontology hierarchies in XPath queries

Multimedia big data computing for in-depth event analysis

While the most part of ”big data” systems target text-based analytics, multimedia data, which makes up about 2/3 of internet traffic, provide unprecedented opportunities for understanding and responding to real world situations and challenges. Multimedia Big Data Computing is the new topic that focus on all aspects of distributed computing systems that enable massive scale image and video analytics. During the course of this paper we describe BPEM (Big Picture Event Monitor), a Multimedia Big Data Computing framework that operates over streams of digital photos generated by online communities, and enables monitoring the relationship between real world events and social media user reaction in real-time. As a case example, the paper examines publicly available social media data that relate to the Mobile World Congress 2014 that has been harvested and analyzed using the described system.Peer ReviewedPostprint (author's final draft

Towards the cloudification of the social networks analytics

In the last years, with the increase of the available data from social networks and the rise of big data technologies, social data has emerged as one of the most profitable market for companies to increase their benefits. Besides, social computation scientists see such data as a vast ocean of information to study modern human societies. Nowadays, enterprises and researchers are developing their own mining tools in house, or they are outsourcing their social media mining needs to specialised companies with its consequent economical cost. In this paper, we present the first cloud computing service to facilitate the deployment of social media analytics applications to allow data practitioners to use social mining tools as a service. The main advantage of this service is the possibility to run different queries at the same time and combine their results in real time. Additionally, we also introduce twearch, a prototype to develop twitter mining algorithms as services in the cloud.Peer ReviewedPostprint (author’s final draft

Diferencias entre pacientes de Parkinson y controles en la conducta motora fina basada en la información visual y propioceptiva versus solo propioceptiva

Personality differences based on fine motor precision performance were studied in early stage Parkinson’s patients and an age-matched control group under two different test conditions: Proprioceptive + visual information and proprioceptive information alone. A comparative data analysis for deviations of three measured movement types (transversal, frontal and sagittal) was done for both hands (dominant and non-dominant) with relation to personality di-mensions. There were found significant differences between the two groups in decision making dimension and emotionality. After splitting the data for gender subgroups, some significant differences were found for men but not for women. The differences in fine motor task performance varied, being better in some directions for the Parkinson’s patients and worse in others. The findings may suggest that medication has both positive and negative effects on motor per-formance and provoke personality changes, being more pronounced in men.Las diferencias de personalidad basadas en la precisión de la ejecución motora fina fueron estudiadas en pacientes con Parkinson en la etapa inicial de la enfermedad y en un grupo control de edad similar bajo dos con-diciones de ensayo diferentes: información propioceptiva + visual y solo información propioceptiva. Se hizo un análisis de datos comparativo de las desviaciones en los tres tipos de movimiento medidos (transversal, frontal y sagital) con las dos manos (dominante y no dominante) y en relación a las dimensiones de la personalidad. Se encontraron diferencias significativas entre los dos grupos en las dimensiones toma de decisiones y emotividad. Después de dividir los datos en subgrupos según el género, se encontraron algunas diferencias significativas para los hombres pero no para las mujeres. Las diferencias en el rendimiento en estas tareas variaron, siendo mejor en algunas direcciones para los pacientes de Parkinson y peor en otras. Los hallazgos sugieren que la medicación puede tener efectos tanto positivos como negativos en el rendimiento motor y provocar cambios en la personalidad, siendo más pronunciados en los hombre

Distributed training of deep neural networks with spark: The MareNostrum experience

Deployment of a distributed deep learning technology stack on a large parallel system is a very complex process, involving the integration and configuration of several layers of both, general-purpose and custom software. The details of such kind of deployments are rarely described in the literature. This paper presents the experiences observed during the deployment of a technology stack to enable deep learning workloads on MareNostrum, a petascale supercomputer. The components of a layered architecture, based on the usage of Apache Spark, are described and the performance and scalability of the resulting system is evaluated. This is followed by a discussion about the impact of different configurations including parallelism, storage and networking alternatives, and other aspects related to the execution of deep learning workloads on a traditional HPC setup. The derived conclusions should be useful to guide similarly complex deployments in the future.Peer ReviewedPostprint (author's final draft

Reliable scholarly objects search and interchange framework

Authors of scholarly objects might fear that there is a potential risk that the original material they publish in online sites or that they submit for evaluation to scientific journals or conferences is used by others as their own material. In such cases, it would not be easy for the original authors to prove authorship of the original contribution. In similar circumstances, it is very difficult to prove the authorship or origin of some materials that are being distributed amongst social networks, private or institutional websites or any other means through the Internet, namely documents, papers, images, data, etc. Those materials can be easily plagiarised (e.g. partially or totally translated) and redistributed without any control and with no means to prove authorship. In this context, we propose an online framework for the registration, search, interchange and trade of scholarly objects, which helps to overcome the potential drawbacks of online distribution and publishing. This framework acts as an intellectual property repository and sales point, where people is able to register content and determine the way they want to trade it, while providing innovative search capabilities based on the MPEG Query Format standard [1]. Creative Commons (CC) [2] limitations are identified and overcome by means of a licensing approach that combines Rights Expression Languages and the MPEG-21 Media Value Chain Ontology [3].Peer ReviewedPostprint (published version

Reliable scholarly objects search and interchange framework

Authors of scholarly objects might fear that there is a potential risk that the original material they publish in online sites or that they submit for evaluation to scientific journals or conferences is used by others as their own material. In such cases, it would not be easy for the original authors to prove authorship of the original contribution. In similar circumstances, it is very difficult to prove the authorship or origin of some materials that are being distributed amongst social networks, private or institutional websites or any other means through the Internet, namely documents, papers, images, data, etc. Those materials can be easily plagiarised (e.g. partially or totally translated) and redistributed without any control and with no means to prove authorship. In this context, we propose an online framework for the registration, search, interchange and trade of scholarly objects, which helps to overcome the potential drawbacks of online distribution and publishing. This framework acts as an intellectual property repository and sales point, where people is able to register content and determine the way they want to trade it, while providing innovative search capabilities based on the MPEG Query Format standard [1]. Creative Commons (CC) [2] limitations are identified and overcome by means of a licensing approach that combines Rights Expression Languages and the MPEG-21 Media Value Chain Ontology [3].Peer ReviewedPostprint (published version

How MPEG Query Format enables advanced multimedia functionalities

In December 2008, ISO/IEC SC29WG11 (more commonly known as MPEG) published the ISO/IEC 15938-12 standard, i.e. the MPEG Query Format (MPQF), providing a uniform search&retrieval interface for multimedia repositories. While the MPQF’s coverage of basic retrieval functionalities is unequivocal, it’s suitability for advanced retrieval tasks is still under discussion. This paper analyzes how MPQF addresses four of the most relevant approaches for advanced multimedia retrieval: Query By Example (QBE), Retrieval trough Semantic Indexing, Interactive Retrieval, and Personalized and Adaptive Retrieval. The paper analyzes the contribution of MPQF in narrowing the semantic gap, and the flexibility of the standard. The paper proposes several language extensions to solve the different identified limitations. These extensions are intended to contribute to the forthcoming standardization process of the envisaged MPQF’s version 2.Postprint (author’s final draft